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Abstract 


The  Joint  Advanced  Distributed  Simulation  (JADS)  Joint  Test  and  Evaluation  (JT&E)  was 
chartered  by  the  Office  of  the  Secretary  of  Defense  (Acquisition  and  Technology)  in  October 
1 994  to  investigate  the  utility  of  advanced  distributed  simulation  (ADS)  technologies  for  support 
of  test  and  evaluation  (T&EI  The  JADS  Joint  Test  Force  (JTF)  conducted  a  System  Integration 
Test  (SIT)  in  which  ADS  was  used  to  support  the  testing  of  an  integrated  missile  weapon/launch 
aircraft  system  in  operationally  realistic  scenarios.  The  SIT  scenarios  simulated  a  single  shooter 
aircraft  launching  an  air-to-air  missile  against  a  single  target  aircraft.  Extensive  testing  was 
performed  involving  two  different  ADS  architectures: 

•  A  linked  laboratory  configuration  in  which  the  shooter  and  target  were  represented  by  manned 
flight  laboratories  and  the  missile  by  an  AIM-9M  Sidewinder  hardware-in-the-loop  (HWIL) 
laboratory. 

•  A  live  shooter/target  configuration  in  which  the  shooter  and  target  were  represented  by  live  F- 
16  fighters  and  the  missile  by  an  AIM- 120  Advanced  Medium  Range  Air-to-Air  Missile 
(AMRAAM)  HWIL  laboratory. 

Testing  was  completed  in  October  1997,  and  evaluation  of  the  results  supports  the  conclusion  that 
each  ADS  configuration  has  utility  for  T&E  of  the  corresponding  air-to-air  missile  involved. 

This  paper  discusses  the  following: 

•  Description  of  the  two  different  architectures  utilized  in  the  SIT  testing. 

•  Technical  challenges  in  implementing  ADS. 

•  Lessons  learned  from  implementing  ADS. 

•  Conclusions  on  the  utility  of  ADS-based  testing  of  air-to-air  missiles. 

•  Benefits  of  implementing  ADS-based  testing  of  air-to-air  missiles. 

Overview 


The  JADS  JT&E  program  was  chartered  by  the  Office  of  the  Secretary  of  Defense  in  October 
1994  to  investigate  the  utility  of  ADS  technologies  for  support  of  T&E.  The  JADS  JTF  is  Air 
Force  led,  with  Army  and  Navy  participation,  and  is  scheduled  for  completion  in  1999.  This 
paper  describes  the  results  from  the  first  of  three  separate  JADS  tests,  the  SIT,  which  was 
completed  in  October  1997. 

The  SIT  investigated  the  ability  of  ADS  to  support  air-to-air  missile  testing.  The  test  included 
two  sequential  phases,  a  Linked  Simulators  Phase  (LSP)  and  a  Live  Fly  Phase  (LFP).  Both 
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phases  incorporated  one-versus-one  scenarios  based  upon  profiles  flown  during  live  test  activities 
and  limited  target  countermeasures  capability. 

Linked  Simulators  Phase  (LSP) 

The  LSP  was  executed  by  the  JADS  JTF  and  the  Naval  Air  Warfare  Center,  Weapons  Division 
(NAWCWPNS)  between  August  and  November  1996.  The  one-versus-one  scenario  utilized  in 
the  LSP  missions  was  taken  from  previous  AIM-9M  testing. 

LSP  Architecture 


In  the  LSP,  the  shooter,  target,  and  missile  were  all  represented  by  simulation  laboratories.  ADS 
techniques  were  used  to  link  NAWCWPNS  manned  flight  laboratories  representing  the  aircraft  to 
an  air-to-air  missile  hardware-in-the-loop  (HWIL)  laboratory  representing  the  missile.  The  LSP 
test  configuration  is  shown  in  Figure  1.  The  F/A-18  Weapon  System  Support  Facility  (WSSF)  at 
China  Lake,  California,  and  the  F-14D  Weapon  System  Integration  Center  (WSIC)  at  Point 
Mugu,  California,  were  the  shooter  and  target,  respectively.  These  laboratories  were  linked  to 
each  other  and  to  an  AIM-9M-8/9  HWEL  laboratory  at  the  Simulation  Laboratory  (SIMLAB)  at 
China  Lake.  Interfacing  of  the  laboratories  to  the  network  was  by  means  of  network  interface 
units  (NIUs).  The  launch  aircraft  laboratory  “fired”  the  AIM-9  in  the  SIMLAB  at  the  simulated 
target  aircraft,  and  the  AIM-9  seeker  responded  to  infrared  (IR)  sources  in  the  SIMLAB  which 
simulated  the  IR  signatures  and  relative  motions  of  the  target  aircraft  and  the  flare 
countermeasures.  Real-time  links  between  the  laboratories  allowed  the  players  to  respond  to  each 
other. 

The  nodes  exchanged  entity  state  information  with  each  other  by  means  of  distributed  interactive 
simulation  (DIS)  protocol  data  units  (PDUs).  However,  the  stores  management  system  (SMS) 
data  exchange  between  the  F/A-18  WSSF  and  the  AIM-9  SIMLAB  used  the  tactical  military 
standard  1553  protocol  because  no  suitable  DIS  protocol  exists  for  these  data,  because  this 
exchange  was  only  between  the  WSSF  and  the  SIMLAB,  and  because  use  of  the  tactical  protocol 
was  appropriate  for  integrated  weapon  system  testing. 

The  test  runs  were  controlled  initially  by  the  Battle  Management  Interoperability  Center  (BMIC) 
at  Point  Mugu  and  later  by  the  Test  Control  and  Analysis  Center  (TCAC)  in  Albuquerque,  New 
Mexico.  The  control  center  ensured  that  all  nodes  were  ready  for  each  run  and  issued  the 
commands  to  start  and  stop  the  runs.  PDUs  were  processed  at  the  control  center  to  provide 
JADS  test  controllers  and  analysts  with  real-time  stealth  node  viewing  of  the  simulated 
engagement. 
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Figure  1.  Linked  Simulators  Phase  Test  Configuration 


LSP  Results 

LSP  results  were  documented  in  the  final  report  for  that  phase  (Ref  1)  and  other  technical  papers 

(Refs.  2  through  4).  Key  findings  were  as  follows. 

•  The  SIMLAB  missile  performance  was  valid  for  its  target  representation. 

•  The  manual  method  for  replicating  a  given  profile  resulted  in  very  good  run-to-run 
reproducibility  of  the  engagement. 

•  The  average  latency  of  all  entity  state  data  was  relatively  small  (<100  milUseconds  from 
simulation  to  simulation).  However,  relatively  large  latency  variations  were  observed  which 
introduced  uncertainty  in  the  target  position  (Ref  4).  The  random  nature  of  these  variations 
comphcates  evaluation  of  closed-loop  interactions  between  the  target  and  the  missile. 

•  The  network  provided  ample  bandwidth. 

•  There  were  no  significant  ADS-induced  errors. 

•  DIS  PDUs  were  adequate  for  entity  state  data  exchanges. 

•  The  centralized  test  control  procedures  worked  well. 


Live  Fly  Phase  (LFP) 


The  LFP  was  executed  by  the  JADS  JTF  and  the  46*^  Test  Wing  at  Eglin  Air  Force  Base  (AFB), 
Florida,  during  1997.  The  one-versus-one  scenarios  utilized  in  the  LFP  missions  were  taken  from 
previous  AMRAAM  testing. 

LFP  Architecture 


In  the  LFP,  the  shooter  and  target  were  represented  by  live  aircraft  and  the  missile  by  a  simulator. 
ADS  techniques  were  used  to  link  two  live  F-16  fighter  aircraft  flying  over  the  Eglin  Gulf  Test 
Range  to  the  AMRAAM  AIM-120  Missile  Simulation  Laboratory  (MISILAB)  HWIL  simulation 
facility  at  Eglin  AFB.  The  LFP  test  configuration  is  shown  in  Figure  2. 


LIVE  AIRCRAFT 

^16  SHOOTER 


CC>NTROi/COO  RO 


MISSILE 
AMRAAM  MISILAB^ 


Figure  2.  Live  Fly  Phase  Test  Configuration 


Global  positioning  system  (GPS)  and  telemetry  (TM)  data  were  downlinked  from  the  aircraft  and 
passed  to  the  Central  Control  Facility  (CCF)  at  Eglin  AFB.  GPS,  inertial  navigation  system 
^S),  and  tracking  radar  data  for  each  aircraft  were  combined  by  the  time-space-position 
information  (TSPI)  Data  Processor  (TDP)  in  the  CCF  to  produce  optimal  entity  state  solutions. 
The  aircraft  entity  state  data  were  transformed  into  DIS  PDUs  and  transferred  to  the  AMRAAM 
HWIL  laboratory  at  the  MISILAB  over  a  T3  link  (with  a  bandwidth  of  about  45  megabits  per 
second).  The  shooter  aircraft  “fired”  the  AMRAAM  in  the  MISILAB  at  the  target  and  provided 
data  link  updates  of  the  target  position  and  velocity  to  the  missile  during  its  flyout.  The 
AMRAAM  seeker  was  mounted  on  a  flight  table  and  responded  to  radio  frequency  (RF)  sources 
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in  the  MTSTT  -AR  which  simulated  the  seeker  return  from  the  target,  the  relative  motions  of  the 
target  and  the  missile,  and  electronic  countermeasures  (ECM).  ATI  link  (with  a  bandwidth  of 
about  1.5  megabits  per  second)  between  the  CCF  and  the  JADS  TCAC  allowed  JADS  personnel 
to  nionitor  the  simulated  intercepts. 

The  actual  umbilical  and  data  link  messages  from  the  shooter  aircraft  were  used  to  initialize, 
launch,  and  update  the  missile  in  the  MISILAB  during  each  simulated  engagement.  The  shooter 
carried  a  pod  which  emulated  the  AMRAAM  missile  in  its  prelaunch  configuration,  and 
AMRAAM  telemetry  from  the  pod  was  downlinked  and  processed  by  the  CCF.  The  telemetry 
was  converted  into  DIS  PDUs  and  transferred  to  the  MISILAB  over  the  T3  link.  The  messages 
were  then  reconstructed  and  synchronized  to  the  aircraft  TSPI  data  by  the  Advanced  Aircraft 
Simulation  Interface  (AASI)  in  the  MISILAB. 

The  test  runs  were  controlled  from  the  CCF.  The  control  center  ensured  that  all  players  were 
ready  for  each  run  and  issued  the  commands  to  start  and  stop  the  passes.  PDUs  were  processed 
at  the  TCAC  to  provide  JADS  personnel  with  real-time  stealth  node  viewing  of  the  simulated 
engagement. 

LFP  Results 


LFP  testing  was  completed  in  October  1997,  and  results  were  documented  in  the  final  report  for 
that  phase  (Ref  5)  and  other  technical  papers  (Refs.  6  through  10).  Key  findings  were  as  follows. 

•  The  MISILAB  missile  performance  was  valid  for  its  target  presentation,  and  the  target 
presentation  accurately  represented  the  live  target.  Also,  the  umbilical  and  data  link  messages 
provided  to  the  MISILAB  missile  accurately  replicated  those  generated  by  the  shooter. 

•  The  TDP  was  able  to  provide  the  required  accuracy  of  aircraft  TSPI.  The  TDP  solutions 
were  estimated  to  be  accurate  to  within  1-3  meters  in  position  and  1  meter  per  second  in 
velocity.  These  values  met  the  MISILAB  accuracy  requirements. 

-  The  use  of  multiple  TSPI  inputs  resulted  in  robust  TDP  performance.  In  particular, 
periodic  GPS  dropouts  did  not  significantly  degrade  the  accuracy  of  the  position  solution, 
because  the  TDP  used  the  accurate  INS  data  to  propagate  the  solution  between  GPS 
updates. 

•  The  data  required  to  drive  the  MISILAB  simulation  were  properly  synchronized  before  being 
input. 

-  Variable  processing  delays  resulted  in  aircraft  entity  state  data  and  the  umbilical  and  data 
Unk  messages  arriving  at  the  MISILAB  in  an  unsynchronized  fashion. 

-  Buffering  and  time  alignment  of  the  data  resulted  in  synchronized  inputs. 

•  Significant  latencies  resulted  from  the  LFP  configuration. 

-  Processing  of  the  TSPI  data  by  the  TDP  and  by  a  post-TDP  smoother  resulted  in  latencies 
of  about  2.4  seconds  for  aircraft  entity  state  data  arriving  at  the  MISILAB.  Smoothing  of 
the  TDP  solution  was  required  for  proper  MISILAB  simulation  performance. 

Buffering  of  die  data  for  synchronization  to  the  MISILAB  simulation  resulted  in  an 
additional  600  milliseconds  of  latency. 

Total  latency  of  the  missile  simulation  was  about  3  seconds  relative  to  the  live  aircraft. 
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•  There  was  good  network  performance  with  ample  bandwidth,  no  ADS-induced  errors,  and  no 
wide  area  network  failures. 

Lessons  Learned 


Lessons  learned  from  SIT  testing  can  be  summarized  as  follows. 

•  For  T&E  applications,  the  technology  is  not  at  the  "plug-and-play"  stage.  While  practical  and 
cost  effective  in  many  cases,  implementation  is  more  challenging  than  many  people  think. 

Plan  for  a  lot  of  rehearsals  and  "fix"  time. 

•  The  architecture  build-up  must  be  incremental,  beginning  with  check  out  of  the  ADS  elements 
in  a  stand-alone  mode,  and  evolving,  step  by  step,  to  the  fully  integrated  configuration. 

•  Problem  solving/fixes  frequently  require  verification  in  a  full-up  network  configuration. 

•  Configuration  control  in  a  distributed  environment  is  difficult,  but  essential. 

•  The  effects  of  latency  and  olher  ADS-induced  errors  can  often  (not  always)  be  mitigated. 

•  Data  synchronization  is  as  much  a  challenge  as  latency  management.  Synchronization  is  best 
based  on  a  “master  clock”  built  into  the  architecture. 

•  Instrumentation  and  data  management  are  more  complex  in  a  distributed  test. 

•  Linking  may  require  special  purpose  interfaces,  and  their  development  must  be  plarmed  for. 

Utility  Conclusions 

An  LSP  type  architecture  involving  all  simulation  laboratories  has  utility  for 

•  Missile  weapon/launch  aircraft  system  integration  T&E. 

•  Parametric  studies,  due  to  good  manual  reproducibility  of  the  profile. 

•  Rehearsal  and  refinement  of  live  fire  engagement  scenarios. 

An  LFP  type  architecture  involving  a  live  shooter  and  target  has  utility  for 

•  Missile  weapon/launch  aircraft  system  integration  T&E,  especially  evaluation  of  the  targeting 
messages  supplied  to  the  missile  by  the  shooter. 

•  Rehearsal  and  refinement  of  live  fire  engagement  scenarios. 

•  Tactics  development  involving  closed-loop  interactions  between  the  shooter  and  target. 

•  Efficient  testing  utilizing  an  analyst-in-the-loop  for  timely  feedback  during  the  mission. 

ADS  Benefits 


The  benefits  of  ADS-supported  testing  are  best  realized  when  this  technique  is  added  to  a  total 
weapon  system  testing  program.  ADS-supported  tests  are  not  meant  to  replace  any  of  the  current 
testing  techniques,  including  live  fire  tests,  but  rather  to  supplement  current  techniques  and 
provide  a  more  comprehensive  evaluation  of  a  weapon  system.  When  the  appropriate  mix  of 
testing  techniques  is  used,  the  following  benefits  are  realized  from  the  addition  of  ADS-supported 
testing: 

•  Cost  savings  benefits. 

A  missile  testing  program  which  uses  ADS-supported  tests  to  supplement  live  fire  tests 
can  be  more  cost  effective  than  live  fire  testing  alone.  In  a  limited  number  of  cases, 
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relatively  inexpensive  ADS-supported  tests  can  replace  costly  live  tests.  Generally,  live 
tests  are  not  replaced;  instead,  the  proper  use  of  ADS  can  result  in  a  higher  success  rate 
for  the  live  tests  by  identifying  failures  before  the  fact  (cost  avoidance)  and  can  aid  in  the 
optimal  selection  of  live  test  scenarios  and  associated  measures.  A  methodology  for 
determining  the  cost  benefit  of  mixing  ADS  missions  into  a  missile  testing  program  is 
outlined  in  Appendix  B  of  Reference  1 1,  along  with  an  example  of  applying  the 
methodology  to  one  phase  of  AMRAAM  testing. 

•  Improved  testing  benefits. 

-  Testing  using  a  linked  laboratory  ADS  architecture  (similar  to  the  LSP  architecture)  is 
more  reproducible  than  live  fire  testing,  because  scenario  conditions  are  more  readily 
controlled  and  trials  can  be  replayed  for  additional  missile  responses.  This  allows  more 
trials  to  be  combined  for  analysis,  giving  greater  confidence  in  evaluation  results. 

-  ADS-supported  testing  allows  the  evaluation  of  certain  classified  techniques  in  which  the 
ECM  device  cannot  be  permitted  to  radiate  its  RF  emission  on  an  open  range.  Rather,  the 
ECM  emissions  can  be  restricted  to  the  missile  HWIL  laboratory  where  they  are  screened 
from  unauthorized  observation  and  where  the  effects  of  the  ECM  on  missile  performance 
can  be  immediately  observed  by  analysts. 

ADS  allows  the  force  density  of  the  scenario  to  be  increased.  The  number  of  friendly  and 
threat  systems  can  be  increased  by  representing  them  with  either  manned  laboratories  (if 
realistic  man-in-the-loop  control  of  the  systems  is  needed)  or  digital  models  (if  scripted 
behavior  is  acceptable).  The  inability  to  evaluate  system  performance  in  combat- 
representative  environments  is  a  common  limitation  in  operational  testing  and  an  area  in 
which  ADS  can  improve  the  operational  test  environment  (Ref  12). 

-  ADS-supported  tests  exhibit  more  realism  than  either  analytical  simulation  models 
(because  actual  hardware  is  used)  or  stand-alone  HWIL  laboratories  (because  realistic 
shooter  and  target  inputs  are  provided). 

•  More  efficient  testing  benefits. 

-  Testing  using  a  hve  shooter-target  ADS  architecture  (similar  to  the  LFP  architecture)  is 
more  efficient  than  live  fire  testing  because  the  analysts  get  immediate  feedback  on  each 
pass  of  a  multiple  pass  mission.  This  allows  adjustments  to  be  made  to  the  remaining  test 
matrix,  if  necessaiy,  while  the  live  shooter  and  target  platforms  are  still  on  range.  This 
“analyst-in-the-loop”  feature  of  ADS  testing  would  be  especially  useful  in  efficiently 
progressing  through  an  ECM  testing  matrix  which  involves  varying  a  number  of  ECM- 
related  parameters. 

-  Live  fire  tests  can  be  realistically  rehearsed  using  ADS.  This  would  ensure  the  proper 
setup  of  the  scenario  and  reduce  wasted  live  fire  attempts  in  which  the  proper  scenario 
conditions  are  not  achieved.  This  use  of  ADS  would  also  reduce  the  risk  of  a  live  fire 
testing  program  by  identifying  scenarios  which  cannot  be  correctly  executed  or  which 
cannot  achieve  the  stated  objectives  (Ref  12). 


Summary 
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The  SIT  investigated  the  application  of  two  different  ADS  architectures  to  air-to-air  missile 
testing.  Both  architectures  produced  valid  T&E-quality  missile  performance  data.  The  SIT 
results  support  the  conclusion  that  ADS-supported  missile  testing  has  utility  in  a  number  of  areas. 
Further,  there  are  benefits  to  adding  ADS-supported  tests  to  a  total  weapon  system  testing 
program,  including  possible  cost  savings. 

ADS  has  great  potential  as  a  T&E  support  tool;  it  is  a  valuable  addition  to  the  tester’s  tool  kit. 
ADS  will  not  obviate,  but  in  some  cases  may  reduce,  the  need  for  hve  testing. 
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